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Abstract 

We consider the the problem of tracking heavy hitters and quantiles in the distributed streaming model. The 
heavy hitters and quantiles are two important statistics for characterizing a data distribution. Let A be a multiset of 
elements, drawn from the universe U = {1, . . . , u}. For a given < <j> < 1, the 0-heavy hitters are those elements 
of A whose frequency in A is at least 4>\A\; the </>-quantile of A is an element x of U such that at most 4>\A\ elements 
of A are smaller than A and at most (1 — 4>)\A\ elements of A are greater than x. Suppose the elements of A are 
received at k remote sites over time, and each of the sites has a two-way communication channel to a designated 
coordinator, whose goal is to track the set of 0-heavy hitters and the <j!>-quantile of A approximately at all times with 
minimum communication. We give tracking algorithms with worst-case communication cost 0(k/e ■ log n) for both 
problems, where n is the total number of items in A, and e is the approximation error. This substantially improves 
upon the previous known algorithms. We also give matching lower bounds on the communication costs for both 
problems, showing that our algorithms are optimal. We also consider a more general version of the problem where 
we simultaneously track the ^-quantiles for all < (f> < 1. 

1 Introduction 

Data streams have been studied in both the database and theory communities for more than a decade [2, 3]. In this 
model, data items arrive in an online fashion, and the goal is to maintain some function / over all the items that have 
already arrived using small space. A lot of /'s have been considered under the streaming model. The theory community 
have studied various frequency moments [2, 20, 31], geometric problems [1, 19, 30], and some graph problems [5, 15]. 
While the database community have mostly focused on maintaining the frequent items (a.k.a. heavy hitters) [8, 10, 
22,25,26] and quantiles [9, 17, 18], two very important statistics for characterizing a data distribution. Since we 
cannot afford to store all the items, we can only maintain an approximate / (except for some trivial /'s), and all the 
results in the streaming model are expressed as a tradeoff between the approximation error e and the space used by 
the algorithm. After a long and somehow disorganized line of research, the heavy hitter problem is now completely 
understood with both space upper and lower bounds determined at 9(l/e); please see the recent paper by Cormode 
and Hadjieleftheriou [8] for a comprehensive comparison of the existing algorithms for this problem, both theoretically 
and empirically. For maintaining quantiles, the best upper bound is due to a sketch structure by Greenwald and Khanna 
[18], using space 0(1/ e ■ log(en)) where n is the number of items in the stream. This is conjectured to be optimal but 
not yet proved. 

Recent years have witnessed an increasing popularity of another model more general than the streaming model, 
where multiple streams are considered. In this model, multiple streams are received at multiple distributed sites, and 
again we would like to continuously track some function / over the union of all the items that have arrived across 
all the sites. Here the most important measure of complexity is the total communication cost incurred during the 
entire tracking period. This model, which is either referred to as the distributed streaming model or the continuous 
communication model, is a natural combination of the classical communication model [32] and the data stream model. 
Recall that the communication model studies the problem of computing some function / over distributed data using 
minimum communication. The data is predetermined and stored at a number of sites, which communicate with a 
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central coordinator, and the goal is to do a one-time computation of the function /. Thus the distributed streaming 
model is more general as we need to maintain / continuously over time as items arrive in a distributed fashion. 

The rising interest on the distributed streaming model is mainly due to its many applications in distributed 
databases, wireless sensor networks, and network monitoring. As a result, it has attracted a lot of attention lately 
in the database community, resulting in a flurry of research in this area [4,6,7, 12-14, 16,23,24,27-29]. However, 
nearly all works in this area are heuristic and empirical in nature, with a few exceptions to be mentioned shortly. For 
many fundamental problems in this model, our theoretical understandings are still premature. This is to be contrasted 
with the standard streaming model, where theory and practice nicely blend, and in fact many of the most practically 
efficient solutions are the direct products of our theoretical findings. In this paper, we take an important step towards 
an analytical study of the distributed streaming model, by considering the worst-case communication complexity of 
tracking heavy hitters and quantiles, arguably two of the most fundamental problems on data streams. 

The distributed streaming model. We now formally define the distributed streaming model, which is the same as 
in most works in this area. Let A = (ai, . . . , a n ) be a sequence of items, where each item is drawn from the universe 
U = {1, . . . , u}. The sequence A is observed in order by k > 2 remote sites Si,...,Sk collectively, i.e., item a; is 
observed by exactly one of the sites at time instance t%, where t\ < ti < • • • < t n . Let A(t) be the multiset of items 
that have arrived up until time t from all sites. Then the general goal is to continuously track f(A(t)) for some function 
/ at all times t with minimum total communication among the sites. Note that in the classical communication model, 
the goal is to just compute f(A(+oo)); in the data stream model, the goal is to track f(A(t)) for all t but there is only 
one site (fc = 1), and we are interested in the space complexity of the tracking algorithm, not communication. Thus, 
the distributed streaming model is a natural combination of the two, but is also significantly different from either. 

We define the manner of communication more precisely as follows. There is a distinguished coordinator C, who 
will maintain (an approximate) f(t) at all times. There is a two-way communication channel between the coordinator 
and each of the k sites, but there is no direct communication between any two sites (but up to a factor of 2, this is 
not a restriction). Suppose site Sj receives the item a, at time fcj. Based on its local status, Sj may choose to send a 
message to C, which in turn may trigger iterative communication with other sites. We assume that communication is 
instant. When all communication finishes, all the sites who have been involved may have new statuses, getting ready 
for the next item aj+i to arrive. We will measure the communication cost in terms of words, and assume that each 
word consists of 0(log u) = 0(log n) bits. Finally we assume that n is sufficiently large (compared with k and 1 /e); 
if n is too small, a naive solution that transmits every arrival to the coordinator would be the best. 

In this paper we will focus on the communication cost (or simply the cost). Nevertheless, all the algorithms 
proposed in this paper can be implemented both space- and time-efficiently. 

Heavy hitters and quantiles. By taking different /'s, we arrive at different continuous tracking problems. The no- 
tion of e-approximation also differs for different functions. We adopt the following agreed definitions in the literature. 
In the sequel, we abbreviate A(t) as A when there is no confusion. 

For any x € U, let m x (A) be the number of occurrences of x in A. For some user specified < <j> < 1, the set 
of (p-heavy hitters of A is 7^0(^4) = {x | m x (A) > <fi\A\}, where \A\ denotes the total number of items in A. If an 
e-approximation is allowed, then the returned set of heavy hitters must contain (A) and cannot include any x such 
that m x (A) < (</> — e)| A|. If (<j> — e)\A\ < m x (A) < 4>\A\, then x may or may not be reported. In the heavy hitter 
tracking problem, the coordinator should always maintain an approximate Ji^ (A) at all times for a given <j>. 

For any < <j> < 1, the <fi-quantile of A is some x £ U such that at most (j>\A\ items of A are smaller than x and at 
most (1 — 4>)\A\ items of A are greater than x. The quantiles are also called order statistics in the statistics literature. 
In particular, the i-quantile is also known as the median of A. If an e-approximation is allowed, we can return any 
^'-quantile of A such that — e <</>'<</> + e. In the <fi-quantile tracking problem, the coordinator needs to keep 
an e-approximate 0-quantile of A at all times for a given <fi. We also consider a more general version of the problem, 
where we would like to keep track of all the quantiles approximately. More precisely, here the "function" / is a data 
structure from which an e-approximate 0-quantile for any <fi can be extracted. Note that such a structure is equivalent 
to an (approximate) equal-height histogram, which characterizes the entire distribution. 

In particular, from an all-quantile structure, we can easily obtain the (2e)-approximate </>-heavy hitters for any </>, 
as observed in [7]. Therefore, the all-quantile tracking problem is more general than either the 0-heavy hitter tracking 
problem or the 0-quantile tracking problem. In the rest of the paper, we omit the word "approximate" when referring 
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to heavy hitters and quantiles when the context is clear. 

Previous works. Traditionally, query answering in distributed databases follows a "poll" based approach, that is, the 
coordinator collects information from the sites to answer a query posed by the user using minimum communication. 
Such a paradigm falls into the realm of the classical multi-party communication theory. These queries are also referred 
to as one-shot queries in the literature. As long-standing queries that need to be answered continuously become com- 
mon in many modern applications such as sensor network monitoring, network anomaly detection, publish-subscribe 
systems, etc., periodically polling all the sites is neither efficient nor effective (i.e., long latency). Thus, the trend is 
moving towards a "push" based approach [21], in which the sites actively participate in the tracking process. In this 
framework, each site maintains some local conditions, and will not initiate communication unless one of the conditions 
is triggered. Such an approach often leads to much reduced communication overhead compared with the "poll" based 
approach, since the system will react only when "interesting" things are happening. This is the main motivation that 
has led to the distributed streaming model described above. 

Various /'shave been considered under this framework. The simplest case f(A) = A | just counts the total number 
of items received so far across all the sites. This problem can be easily solved with 0(k/e ■ logn) communication 
where each site simply reports to the coordinator whenever its local count increases by a 1 + e factor [23]. The other 
important single-valued statistics are the frequency moments: F p (A) — 'Yl lx {m x (A)) p . Fq is the number of distinct 
items, and can be tracked with cost 0(k/e 2 ■ log n log j) [11]; F2 is the self-join size and can be tracked with cost 
0((fc 2 /e 2 + /c 3 / 2 /e 4 )lognlog^f) [11]. Some heuristic approaches based on predicting future arrivals of items have 
been proposed in [6, 12]. 

Single-valued statistics have very limited expressive power, so multi-valued statistics are often necessary to better 
capture the distribution of data. The most important ones include the heavy hitters and quantiles, and they have also 
been studied under the distributed streaming framework. Babcock and Olston [4] designed some heuristics for the 
top-fc monitoring problem, where the goal is to track the k most frequent items (whose frequency may not be larger 
than Their techniques can be adapted to tracking the heavy hitters [16], but the approach remains heuristic in 

nature. Manjhi et al. [24] also studied the heavy hitter tracking problem, but their communication model and the goal 
are different: They organize the sites in a tree structure and the goal is to minimize the communication only at the 
root node. The all-quantile tracking problem has been studied by Cormode et al. [7], who gave an algorithm with cost 
0(k/e 2 ■ logn). As commented earlier, this also implies a heavy hitter tracking algorithm with the same cost. This 
remains the best communication upper bound for both problems to date. No lower bound is known. 

Our results. Our main results in this paper are the matching upper and lower bounds on the communication cost for 
deterministic algorithms for both the heavy hitter tracking problem and the quantile tracking problem. Specifically, 
we show that for any <fi, both the ^-heavy hitters (Section 2) and the 0-quantile (Section 3) can be tracked with total 
communication cost 0(k/e ■ logn). This improves upon the previous result of [7] by a 0(l/e) factor. We also give 
matching lower bounds for both problems, showing that our tracking protocols are optimal in terms of communication. 
Note that in the classical communication model, we can easily do a one-shot computation of the 0-heavy hitters and 
the 0-quantile easily with cost 0(k/e), as observed in [7]. Interestingly, our results show that requiring the heavy 
hitters and quantiles to be tracked at all times indeed increases the communication complexity, but only by a 0(log n) 
factor. In Section 4, we give an algorithm that tracks all quantiles with cost 0(fc/e-log 2 - log n). Because this problem 
is more difficult than the single-quantile problem, it has the same lower bound of fl(k/e ■ logn) as the latter. Thus, 
our all-quantile tracking algorithm is also optimal up to a 6(polylogi) factor. 

2 Tracking the Heavy Hitters 
2.1 The upper bound 

The algorithm. Let m be the current size of A. First, the coordinator C always maintains Cm, an e-approximation 
of m. This can be achieved by letting each site send its local count every time it has increased by a certain amount (to 
be specified shortly). Each site Sj maintains the exact frequency of each x £ U at site Sj, denoted m x j, at all times. 
The overall frequency of x is m x = 53 . m x ,j. Of course, we cannot afford to keep track of m x exactly. Instead, 
the coordinator C maintains an underestimate C.m Xj j of m x ,j, and sets C.m x = J2j C- m x,j as an estimate of m x . 
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Sj will send its local increment of m x j to C, hence updating Cm x j, from time to time following certain rules to 
be specified shortly. In addition, each site Sj maintains Sj.m, an estimate of m, a counter Sj.A(m), denoting the 
increment of Sj.m since its last communication to C about Sj.m, as well as a counter Sj.A(m x ) for each x, denoting 
the increment of Sj.m x since its last communication to C about m x j. 

We can assume that the system starts with m = k/e items; before that we could simply send each item to the 
coordinator. So when the algorithm initiates, all the estimates are exact. We initialize Sj.A(m) and Sj.A(m x ) for all 
x to be 0. The protocols of tracking the 0-heavy hitters are as follows. 

1. Each site Sj: When a new item of x arrives, Sj.A(m) and Sj.A(nii) are incremented by 1. When Sj.A(m) 
(resp. Sj.A(m x )) reaches (e ■ Sj.m)/3k, site Sj sends a message (all, (e • Sj.m)/3k) (resp. (x, (e ■ Sj.m)/3k)) 
to the coordinator, and resets Sj.A(m) (resp. Sj.A(m x )) to 0. 

2. Coordinator: When C has received a message (all, (e • Sj.m) /3k) or (x, (e ■ Sj.m) /3k), it updates Cm to 
Cm + (e ■ Sj.m) /3k or Cm x to Cm x + (e • Sj.m) /3k, respectively. Once C has received k signals in the 
forms of (all, (e ■ Sj.m) /3k), it collects the local counts from each site to compute the exact value of m, sets 
Cm = m, and then broadcasts Cm to all sites. Then each site Sj updates its Sj.m to m. After getting a new 
Sj.m, Sj also resets Sj.A(m) to 0. 

Finally, at any time, the coordinator C declares an item x to be a 0-heavy hitter if and only if 

C.m T e 

~ F ^ L ><P+^ (1) 
Cm 2 

Correctness. To prove correctness we first establish the following invariants maintained by the algorithm. 

em 

m x — + k < Cm x < m x , (2) 

em 

m — h k < Cm < m. (3) 

o 

The second inequalities of both (3) and (2) are obvious. The first inequality of (2) is valid since once a site Sj gets 
(e • Sj .m)/3k items of x, it sends a message to the coordinator and the coordinator updates Cm x accordingly. Thus 
the maximum error of Cm in the coordinator is at most 2^j=i(~~%r~ ; — 1) — if — ^. The first inequality of (3) 
follows from a similar reason. Combining (2) and (3), we have 

ma; e C.m x m x 1 m x e 

m 3 Cm m 1 — e/3 m 2' 

which guarantees that the approximate ratio C c ,r ^ is within e/2 of thus classifying an item using (1) will not 
generate any false positives or false negatives. 

Analysis of communication complexity. We divide the whole tracking period into rounds. A round start from the 
time when the coordinator finishes a broadcast of Cm to the time when it initiates the next broadcast. Since the 
coordinator initiates a broadcast after Cm is increased by a factor of 1 + X)i=i( e /3fc) = 1 + e/3, the number of 
rounds is bounded by 

log 1+e/3 n = O 

In each round, the number of messages in the form of (all, (e ■ Sj.m) /3k) sent by all the sites is k by the definition 
of our protocol. Since there are 0(log n/e) rounds in total, the number of messages in the form of (all, (e • Sj .m) /3k) 
can be bounded by 0(k/e ■ logn). On the other hand, it is easy to see that total number of messages of the form 
(x, (e • Sj.m)/3k) is no more than the total number of messages of the form (all, (e • Sj.m)/3k). Therefore, the total 
cost of the whole system is bounded by 0(k/e ■ logn). 

Theorem 2.1 For any e < <j) < 1, there is a deterministic algorithm that continuously tracks the (p-heavy hitters and 
incurs a total communication cost of 0(k/e ■ logn). 
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Implementing with small space. In the algorithm described above, we have assumed that each site maintains all 
of its local frequencies Sj.m x exactly. In fact, it is not difficult to see that our algorithm still works if we replace 
these exact frequencies with a heavy hitter sketch, such as the space-saving sketch [26], that maintains the local e'- 
approximate frequencies for all items for some e' — 8(e). More precisely, such a sketch gives us an approximate 
Sj .m x for any x £ U with absolute error at most e' \Sj\, where | Sj | denotes the current number of items received at Sj 
so far. We need to adjust some of the constants above, but this does not affect our asymptotic results. By using such a 
sketch at each site, our tracking algorithm can be implemented in 0(1/ e) space per site and amortized 0(1) time per 
item. 

2.2 The lower bound 

To give a lower bound on the total communication cost that any deterministic tracking algorithm must take, we first 
consider the number of changes that the set of heavy hitters could experience, where a change is defined to be the 
transition of the frequency of an item from above <p\A\ to below (<fi — e)\A\, or the other way round. Then we show 
that to correctly detect each change, the system must exchange at least a certain amount of messages. The following 
lemma could be established by construction. 

Lemma 2.2 For any <fi > 3e, there is a sequence of item arrivals such that the set of heavy hitters in the whole tracking 
period will have f2(logn/e) changes. 

Proof: Set e' = 2e. We construct two groups of I = 1/(20 — e') items each: Sq = {t\,ti, ■ ■ ■ ,U} and Si = 
{ti+i,ti+2, ■ ■ ■ , }■ Since we only care about the total number of changes of the set of heavy hitters during the whole 
tracking period, we temporarily treat the whole system as one big site and items come one by one. We will construct 
an input sequence under which the set of heavy hitters will undergo £!(log n/e) changes. 

We still divide the whole tracking period to several rounds, and let m j denote the total number of items when round 
i starts. The following invariant will be maintained throughout the construction: 

Let b = i mod 2. When round i starts, all items t £ Sb have frequency 0m,, and all items t £ Si-b have 
frequency (</> — e')mj. 

It can be verified that the total frequency of all items is indeed mj. Note that from the start of round i to the end 
of round i, all the non-heavy hitters become heavy hitters, and all the heavy hitters become non-heavy hitters. In 
what follows we only care about the changes of the former type, which lower bounds the number of changes. To 
maintain the invariant for round i + 1, we construct item arrivals as follows. Without loss of generality, suppose 
Si-b = {ti, t2, ■ ■ ■ , ti}. Let (3 = 6 a ■ We first generate /3mj copies of t\, and then /3m, copies of i 2 , . . . , then 
(3rrii copies of ti, in sequence. After these items we end round i and start round i + 1. At this turning point, the total 
number of items is 

m l+ i = rrti + I ■ (3m-i = -m l . 

(p — e 

Now the frequency of each item in the set Si-b is 

{(j) - e')mi + Pnii = <p ■ -——. mi = <f>m l+1 , 
<p — e 

and the frequency of each item in Sb remains the same, that is, </>m, = ((f) — e')mj+i. Now we have restored the 
invariant and can start round i + 1. 

Finally, we bound the number of rounds. Since the total number of items m, increases by a </>/(</> — e') factor in 

each round, the total number of rounds is 9 (log ± n). Consequently, the total number of changes in the set of heavy 

*-«' , 

hitters (from non-heavy hitters to heavy hitters) is / • 6 (log ± n) = ^(^rrj ■ ^fr- logn) = f2(logn/e). □ 

Now we go back to the distributed scenario and consider the cost of communication for "recognizing" each change. 
Because we allow some approximation when classifying heavy hitters and non-heavy hitters, the valid time to report 
a change is actually a time interval, from the time when its frequency just passes (</> — e)\A\ to the time when its 
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frequency reaches <j)\A\. As long as the tracking algorithm signals the change within this interval, the algorithm is 
considered to be correct. Consider the construction in the proof of Lemma 2.2. In round i, the transition interval 
from a non-heavy hitter to a heavy hitter for an item t must lie inside the period in which the (3rrii copies of t arrive. 
Below we will show that in order for the coordinator to signal the change within this period, O(fc) messages have to 
be exchanged in the worst case using an adversary argument. 

Before presenting the lower bound proof, let us be more precise about the computation model. Recall that in 
the introduction, the model forbids a site to spontaneously initiate communication or change its local status; actions 
can only be triggered as a result of the arrival of an item at this site, or in response to the coordinator. Note that for 
deterministic algorithms this is not a restrictive assumption. In our case, since we only care about the frequency of a 
particular item t increasing from rrii to m, + /3m,, we may assume that each site Sj has a triggering threshold rij, 
meaning that Sj will only initiate communication when the number of copies of t received by Sj is rij. When all the 
communication triggered by the arrival of an item finishes, all the sites that have participated are allowed to update 
their triggering thresholds, but the rest of the sites must retain their old thresholds. 

Lemma 2.3 To correctly recognize a change in the heavy hitters under the input constructed in the proof of lemma 2.2, 
any deterministic algorithm has to incur a communication cost ofQ(k). 

Proof: We will construct an adversary who will send the /3m, copies of t to the sites in a way such that at least f2(fc) 
sites must communicate with the coordinator. Since we are dealing with deterministic algorithms, we may assume that 
the adversary knows the triggering thresholds rij at any time. 
Initially, we must have 

fe 

- 1) < Prrn- (4) 

Otherwise, the adversary can send rij — 1 copies to Sj for all j without triggering any communication, and make the 
algorithm miss the change. Therefore there must be some j such that rij < firrii /k + 1 < 2f3nii/k. The adversary first 
sends 2/3rrii/k copies of t to Sj. Sj will then communicate with the coordinator at least once. After the first 2f3rrii/k 
copies, the new triggering thresholds must still satisfy (4). Similarly, there is some rij' < 2/3rrii/k, and the adversary 
will send another 2f3rrii/k copies of t to Sj>. Such a process can be repeated for 2 ^^ fc = Q(k) times, triggering at 
least fi(fc) messages of communication. □ 

The following lower bound follows immediately from Lemma 2.2 and Lemma 2.3, for the reason that the tracking 
algorithm has to correctly and continuously maintain the whole set of heavy hitters. 

Theorem 2.4 Any deterministic algorithm that continuously tracks the (p-heavy hitters has to incur a total communi- 
cation costoffl(k/e ■ log n), for any <p > 3e. 

Remark. Note that our lower bound above is actually lower bound on the number of messages required. Also recall 
that our algorithm in Section 2.1 sends 0(k/e-\ogn) messages and each message if of constant size. Our lower bound 
implies that one cannot hope to reduce the number of messages by making each of them longer. 

3 Tracking the Median 

In this section we first present an algorithm to track any 0-quantile for < <fi < 1. For ease of presentation we describe 
how to track the median (the 1/2-quantile); the generalization to any ^-quantile is straightforward. Then we give a 
matching lower bound. 

3.1 The upper bound 

For simplicity we assume that all the items in A are distinct; issues with ties can be easily resolved by standard 
techniques such as symbolic perturbation. We divide the whole tracking period into O(logn) rounds; whenever |A| 
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doubles, we start a new round. In the following we focus on one round, and show that our median-tracking algorithm 
has a communication cost of 0(k/e). 

Let to be the cardinality of A at the beginning of a round. Note that m is fixed throughout a round and we always 
have to < \A\. The main idea of our algorithm is to maintain a dynamic set of disjoint intervals in the coordinator (by 
maintaining a set of separating items), such that each interval contains between |m and |m items. We first show that 
if we have such a set of intervals, the median can be tracked efficiently. Afterward we discuss how to maintain these 
intervals. 

Let M denote the approximate median that is kept at the coordinator. We maintain two counters C.A(L) and 
C.A(R), counting the number of items that have been received at all sites to the left and the right of M, respectively. 
These two counters are maintained as underestimates with an absolute error at most |m, by asking each site to send 
in an update whenever it has received -^m items to the left or right of M. So the cost of maintaining them is 0(k/e). 

Whenever \C.A(L) - C.A(R)\ > |m, we update M as follows. 

1 . Compute C.L and C.R as the total number of items to the left and the right of M. W.l.o.g., suppose C.L > C.R 
and let d= {C.L- C.R) /2. 

2. Compute a new median M' such that \r(M) — r(M') — d\ < |to where r(M) is the rank of M in A. Update 
M to M'. Note that M' is at most |to items away from the exact median. We will describe how to compute 
such an M' shortly. 

3. Reset C.A(L) and C.A(R) to 0. 

For the correctness of the algorithm, we can show that our tracking algorithm always maintains an approximate 
median that is at most |to + ^to = em items away from the exact median. The first term |to is due to the fact that 
whenever we update M, M is within an error of at most |to to the exact median. The second term ^to accounts 
for the error introduced by the triggering condition \C.A(L) — C.A(R)\ monitored in the coordinator. Note that we 
keep both C.A(L) and C.A(R) within an additive error of at most |to and whenever \C.A(L) — C.A(R)\ > |to, 
we initiate an update. Therefore, the total error introduced is at most 2 • |m + |to = ^to. 

Now we analyze the communication cost. Step 1 could be done by exchanging O(k) messages. For step 2, first 
note that d < em since by the reasoning above, M is still an e-approximate median. Next, we can find M' quickly with 
the help of the set of intervals. We start by finding the first separating item Y\ of the intervals to the left of M, and then 
collect information from all sites to compute the number of items in the interval [Y\, M], say ri\. If \ni — d\ < |m, 
we are done; otherwise we go on to pick the second separating item Y2 to the left of M, and check if «2 — d\ < |to, 
where ?i2 is the number of items in the interval [Y2, M]. It is easy to see that after at most O(l) such probes, we can 
find an item Y such that the rank difference between Y and the exact median is no more than |m. Note that the cost 
of each probe is O(k) thus the total cost of step 2 is O(k). Finally, we update M at most 0(1/ e) times within a single 
round, since each update increases |^4| by at least a factor of 1 + §. To sum up, the total cost of the algorithm within a 
round is 0(k/e) provided that the dynamic set of intervals are maintained. 

Maintaining the set of intervals. When a new round starts, we initialize the set of intervals as follows: Each site 
Sj (1 < j < k) computes a set of intervals, each containing items, where Aj stands for the set of items Sj 
has received, and then sends the set of intervals to the coordinator (by sending those separating items). Then the 
coordinator can compute the rank of any x € U with an error of at most X)j=i mI^'I = ~§2 m > therefore it can 
compute a set of intervals, each of which contains at least |to and at most |m items. After the coordinator has built 
the set of intervals, it broadcasts them to all the k sites, and then computes the exact number of items in each interval. 
The cost of each rebuilding is 0(k/e). 

During each round, each site Sj maintains a counter for each interval as new items arrive. And whenever the local 
counter of items in some interval I has increased by j^m, it sends a message to the coordinator and the coordinator 
updates the count for interval I accordingly. Whenever the count of some interval in the coordinator C reaches |to, 
the coordinator splits the interval into two intervals, each of which containing at least |m and at most |to items. To 
perform such a split, we can again call the rebuilding algorithm above, except that the rebuilding is only applied to the 
interval /, so the cost is only 0(k). 

The correctness of algorithm is obvious. The total communication cost of interval splits is 0(k/e) in each round, 
since there are at most 0(1 /e) splits and each split incurs a communication cost O(k). 
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Theorem 3.1 There is a deterministic algorithm that continuously tracks the e-approximate median (and generally, 
any <f)-quantile (0 < (f> < 1)) and incurs a total communication cost of 0(k/e ■ logn). 

Implementing with small space. Similar to our heavy hitter tracking algorithm, instead of maintaining the intervals 
exactly at each site, we can again deploy a sketch that maintains the approximate e'-quantiles for some e' = 0(e) to 
maintain these intervals approximately. Suppose we use the Greenwald-Khanna sketch [18], then we can implement 
our </>-quantile tracking algorithm with 0(1/ e ■ log(en)) space per site and amortized 0(log n) time per item. 

3.2 The lower bound 

The idea of the proof of the lower bound is similar as that for the heavy hitters. We try to construct a sequence of input 
with the following properties. 

1. The median will change at least il(logn/e) times. 

2. To correctly recognize each update, any deterministic algorithm has to incur a communication cost of il(k). 

Consider the following construction. The universe consists of only two items and 1. We divide the whole tracking 
period to several rounds and let nij be the number of items at the beginning of round i. We maintain the following 
invariant: When round i starts, the frequency of item & is (0.5 — 2e)mi and the frequency of item 1 — b is (0.5 + 2e)rrii, 
where b = i mod 2. This could be done by inserting Q 5 4 f 2e m i copies of b during round i and then start a new round. 
It is easy to see that there will be at least tt(\ogn/e) rounds and the median will change at least once during each 
round, therefore the total number of changes of the median is f2(logn/e). For the second property, we can invoke the 
same arguments as that for Lemma 2.3. Combining the two properties, we have the following. 

Theorem 3.2 Any deterministic algorithm that continuously tracks the approximate median has to incur a total com- 
munication cost oftt(k/e ■ log n). 

4 Tracking All Quantiles 

In this section, we give a tracking algorithm so that the coordinator C always tracks the e-approximate 0-quantiles 
for all < <j> < 1 simultaneously. We will solve the following equivalent problem: The coordinator is required to 
maintain a data structure from which we can extract the rank r(x) for any x G U in A with an additive error at most 
e| A\. We still assume that all items in A are distinct. 

We divide the whole tracking period into 0(log?i) rounds. In each round |A| roughly doubles. We will show that 
the algorithm's cost in each round is 0(k/e ■ log 2 -). The algorithm restarts itself at the beginning of each round, 
therefore the total communication of the algorithm will be 0(k/e ■ log n log 2 -). 

The data structure. Let m be the cardinality of A at the beginning of a round. The data structure is a binary tree 
T with 0(l/e) leaves. The root r of T corresponds to the entire A. It stores a splitting element x r which is an 
approximate median of A, i.e., it divides A into two parts, either of which contains at least (^ — a)\A\ and at most 
(i + a)\A\ items, for some constant < a < i. Then we recursively build r's left and right subtrees on these two 
parts respectively, until there are no more than em/2 items left. It is clear that T has 0(l/e) nodes in total, and has 
height at most h = log i +a | = 0(log -), though it is not necessarily balanced. Each node in T is naturally associated 
with an interval. Let I u be the interval associated with u. Then I r is the entire U ; suppose v and w are it's children, 
then I u is divided into I v and I w by x u . Set 6 = -Sr. Each node u of T is in addition associated with s u , which is 
an underestimate of \ A n I u \ with an absolute error of at most 9m, i.e., \A (~l I u \ — dm < s u < \A (~l I u \. Please see 
Figure 1 for an illustration of the data structure. 

If the coordinator has such a data structure, it is not difficult to see that we can compute the rank of x with an 
absolute error of at most em. For a given x, we first search down the binary tree and locate the leaf v such that x G I v . 
As we go along the root-to-leaf path, whenever we follow a right child, we add up the s u of its left sibling. In the 
end we add up h such partial sums, each contributing an error of at most 6m, totaling dm ■ h = em/2. Finally, since 
\A D it, | < em/2, the sum of all the s u 's for the preceding intervals of x is off by at most em from the actual rank of 
x. 
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Figure 1 : The data structure that can be used to extract the rank of any x with absolute error < em. 



Initialization. At the beginning of each round, we initialize the data structure similarly as in Section 3. Suppose 
the set of items at Sj is Aj. Each site Sj builds its own structure Sj.T, but with e/32 as the error parameter, and 
ships to C. This costs a communication of 0(k/e). Note that Sj.T allows one to extract the rank of any x within Aj 
with an error of e/32 • \Aj |. By querying each Sj.T, the coordinator can compute the rank of any x with an error of 
Si=i ~h. I ^ = ~§2 m ' wn i cn is enough for the coordinator to build its own C.T. In particular, all the splitting elements 
can be chosen to be within a distance of ^ m to the real median. After building C.T, the coordinator broadcasts it to 
all the sites, costing communication O(kfe). Now each site Sj knows how U is subdivided into those G(l/e) intervals 
represented by the binary tree T. Then for each interval I u , it computes \Aj n I u \ and sends the count to C, so that 
the coordinator has all the exact partial sums s u to start with. It is easy to see that the total communication cost for 
initializing the data structure is 0(k/e). 

Maintaining the partial sums. As items arrive, each site Sj monitors all the intervals I u in T. For each /„, every 
time the local count of items in I u at Sj has increased by 8m/ k, it sends an updated local count to C. Thus in the worst 
case, each site is holding (dm/k — 1) items that have not been reported, leading to a total error of at most 6m. The cost 
of these messages can be bounded as follows. When Sj sends a new count for some interval I u , we charge the cost to 
the 6m/ k new items that have arrived since the last message for 0(k/(6m)) each. Since each item contributes to 
the counts of at most h intervals, it is charged O(h) times, so the total cost charged to one item is 0( J^). There are a 
total of 0(m) items in a single round, so the overall cost is 0(kh/6) = 0(k/e ■ log 2 -). 

Maintaining the splitting elements. The maintenance algorithm above ensures that all the s u are within the desired 
error bound. We still need to take care of all the splitting elements, making sure that they do not deviate from the real 
medians too much. Specifically, when we build T, for any u with children v and if, we ensure that 

l\Ani u \<\Ani v \<^\Ani u \. (5) 

This property can be easily established during initialization, since \ A D I u \ > §m for any internal node u of T, and 
we can estimate \A H I v | with an error of -^m. In the middle of the round, we maintain the following condition: 

1 3 

-s u < s v < -s u . (6) 

Recall that s u (resp. s v ) is an estimate of | A D I u \ (resp. | A n I v |) with an error of at most 6m. As long as (6) holds, 
we have ^ ^ 

-(\A n Ju| - 6m) < -s u <s v < \Anl v \+ 6m. 
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Rearranging, 

\Ani v \ > -\Anl u \ - -■ ^m> -\Ani u \ - -■ -\Anl u \ > — |An/ u |, 

for h > 8. (Note that assuming h larger than any constant does not affect our asymptotic results.) Similarly, we also 
have \A fl I v \ < |||^4 n I M |. Thus condition (6) ensures that the height of T is bounded by h = 6 (log |). 

Whenever (6) is violated, we do a partial rebuilding of the subtree rooted at u to restore this condition. If multiple 
conditions are violated at the same time, we rebuild at the highest such node. To rebuild the subtree rooted at u, we 
apply our initialization algorithm, but only for the range I u . This incurs a cost of 0(k ^ AnIu ^ ), since we are essentially 
building a new data structure on | A PI | elements with error parameter e' = em / 1 A n | . After rebuilding, we have 
restored (5) for u and all its descendants. 

It remains to bound the cost of the partial rebuildings. Similarly as before, we can show that when (6) is violated, 
we must have 

|jlnl„| <— |Ani«,|, (7) 

64 
43 

\Ani v \> —\Ani u \, (8) 

64 

assuming h > 16. Note that both \A fl I v \ and \ A fl I u \ may increase. From (5) to (7), \A n I u \ must increase by 
Q(\A (~1 I v \) = Q(\A n 7 U |); from (5) to (8), \A n I v \ must increase by Q(\A R /„(), which implies that \A n I u \ must 
also increase by tt(\A fl I u \) since I v C I u . This means that between two partial rebuildings of u, \A fl I u \ must 
have increased by a constant factor. Thus, we can charge the rebuilding cost of u to the il(\A n I u \) new items that 
have arrived since the last rebuilding, 0(k/(em)) each. Since each item is contained in the intervals of 0(h) nodes, 
it is charged a cost of 0(hk/(em)) in total. Therefore, the total cost of all the partial rebuildings in this round is 
0(hk/e) = (9(fc/e-logi). 

Maintaining the leaves. Finally, we need to make sure that \A fl I v \ < |m for each leaf v as required by the data 
structure. During initialization, we can easily ensure that |em < \A n < |em. During the round, the coordinator 
monitors s v , and will split v by adding two new leaves below v whenever s v > (| — 0)m. Since s v has error at most 
9m, this splitting condition will ensure that \A n I v \ < |m. To split v, we again call our initialization algorithm on 
the interval I v , incurring a cost of 0(k 1^01°! ) = 0(k). Since we create at most 0(l/e) leaves in this entire round, 
the total cost for all the splittings is 0(k/e). 

Putting everything together, we obtain the following result. 

Theorem 4.1 There is a deterministic algorithm that continuously tracks the <p-quantiles for all < 4> < 1 simulta- 
neously and incurs a total communication cost of 0(k/t ■ log n log 2 -). 

Implementing with small space. Similar as before, instead of maintaining the counts in the intervals associated with 
T exactly at each site, we can again deploy a sketch that maintains the approximate e'-quantiles for some e' = 0(9) to 
maintain these intervals approximately. Suppose we use the Greenwald-Khanna sketch [18], then we can implement 
our all-quantile tracking algorithm with 0(1/9 ■ \og(9n)) = 0(1/ e ■ logilog(en)) space per site and amortized 
0(log n) time per item. 



5 Open Problems 

We have restricted ourselves to deterministic algorithms in the paper. If randomization is allowed, simple random 
sampling can be used to achieve a cost of 0((k + 1/e 2 ) • polylog(n, k, 1/e)) for tracking both the heavy hitters and 
the quantiles. This observation has been well exploited in maintaining the heavy hitters and quantiles for a single 
stream when both insertions and deletions are present (see e.g. [17]). This breaks the deterministic lower bound for 
e = u(l/k). It is not known if randomization can still help for smaller e. Deriving lower bounds for randomized 
algorithms is also an interesting open problem. Another possible direction is to design algorithms to track the heavy 
hitters and quantiles within a sliding window in the distributed streaming model. 
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